Extending digital artifacts through an interactive surface to a mobile device and creating a communication channel between a mobile device and a second mobile device via the interactive surface

ABSTRACT

A unique system and method that facilitates extending input/output capabilities for resource deficient mobile devices and interactions between multiple heterogeneous devices is provided. The system and method involve an interactive surface to which the desired mobile devices can be connected. The interactive surface can provide an enhanced display space and customization controls for mobile devices that lack adequate displays and input capabilities. In addition, the interactive surface can be employed to permit communication and interaction between multiple mobile devices that otherwise are unable to interact with each other. When connected to the interactive surface, the mobile devices can share information, view information from their respective devices, and store information to the interactive surface. Furthermore, the interactive surface can resume activity states of mobile devices that were previously communicating upon re-connection to the surface.

RELATED APPLICATION

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 11/166,364, filed Jun. 24, 2005, entitled“Extending Digital Artifacts Through An Interactive Surface,” andincorporated by reference herein.

BACKGROUND

In recent years, portable computing and communication devices haveinundated the market place. Competition has become fierce amongmanufacturers as they compete for the attention of consumers as well astheir dollars. Thus, it is not uncommon for the average consumer to owna variety of portable devices such as a cell phone and a personaldigital assistant (PDA) made from the same or different manufacturers orbrands. Traditionally, different computing devices are not made tocommunicate with each other—especially if manufactured by or sold underdifferent brands. For example, a Brand Q PDA typically cannot “talk” toa Brand Z PDA. Similarly, Brand Q PDA cannot interact with a Brand Zsmartphone. For most users, this can cause many inconveniences andresult in large amounts of wasted time because information must beentered separately into each device.

Associating peripheral input/output devices such as wireless keyboardsand printers with a particular portable device can involve complex andcumbersome pairing procedures as well. For example, Bluetooth technologyprovides one conventional technique for pairing a device with aperipheral input/output tool. Unfortunately, Bluetooth only allows forfixed pairings. When a user of the tool is changed, the pairing processmust be re-done. For instance, imagine that TOM's Bluetooth-enabledkeyboard is paired with his laptop computer. TOM lends MARY his keyboardfor her use with her tablet PC. When MARY is finished with the keyboard,she returns it to TOM. In this brief scenario, the Bluetooth-enabledtool requires the pairing process to be performed 3 separate times:twice by TOM and once by MARY.

In addition to the lack or inability of communication between differentdevices, technology, in general, as well as consumers have converged onthe desire and/or need for smaller and smaller devices. Unfortunately,the smaller the device, the smaller the display area or user interface.Consequently, the amount of information which can be viewed at once orreadily viewed at all may be substantially reduced or non-existent. Forinstance, personalizing the device or mere general use of the device canbe time-consuming and frustrating for the user and result in an overallpoor user experience.

SUMMARY

The following presents a simplified summary in order to provide a basicunderstanding of some aspects of the systems and/or methods discussedherein. This summary is not an extensive overview of the systems and/ormethods discussed herein. It is not intended to identify key/criticalelements or to delineate the scope of such systems and/or methods. Itssole purpose is to present some concepts in a simplified form as aprelude to the more detailed description that is presented later.

The subject application relates to a system(s) and/or methodology thatfacilitate operation of at least one mobile device and/or interactionbetween multiple mobile devices via an interactive surface. Inparticular, one or more mobile devices can connect to the interactivesurface. Once connected, the one or more mobile devices can make use ofthe expanded display space of the interactive surface, thus making itmuch easier for the user to operate, customize, or manipulate data onthe mobile device. Although such operations are particularly well suitedto a horizontal or slightly angled interactive surface, which allowsplacing and letting go of multiple devices on the surface, similarinteractions could be supported on a vertical or otherwise orientedsurface by touching device(s) to the surface and then holding them thereor putting them down elsewhere.

According to a first approach, for example, a mobile device canestablish a wireless connection with the interactive surface. Theinteractive surface comprises a display space that is substantiallylarger than the display of the mobile device. Thus, when connectedthereto, the display of the mobile device can be visualized on anexpanded scale via the interactive surface. A variety of operations thatmay otherwise be performed directly on the mobile device with at leastsome difficulty can now be performed with greater ease via theinteractive surface. Moreover, the interactive surface component acts asa portal to the mobile device display and/or data and hence, the usermaintains control of the mobile device and its contents. To furtherenhance the user's overall experience, the interactive surface can betouch-sensitive and/or include a touch-sensitive pad and/or one or moreother input mechanisms (e.g., keyboard, mouse, stylus, etc.) tofacilitate navigation of data displayed on the interactive surface aswell as on the mobile device.

Connection with the interactive surface can be accomplished in one ormore different manners. For instance, the mobile device can be placed onthe surface. Using a visual pattern located on the mobile device, thesurface can read the pattern and identify the type and/or name of thedevice and call up the appropriate protocol(s) in order to open orcreate the proper channels to communicate and interact with the device.Alternatively, the visual pattern can be read from the mobile devicewhen the mobile device is located within a physical proximity of theinteractive surface. In either case, the connection can be establishedautomatically or at the discretion of the user by way of employing oneor more controls or buttons on the interactive surface. For example,when the presence of a mobile device is detected, a dialogue box mayappear and ask the user whether a connection is desired—to which theuser must respond before the connection is established. Disconnectioncan be accomplished in a similar manner. That is, the device may beremoved from the surface or explicit controls or commands may berequired to disconnect the mobile device from the surface.

Non-visual patterns or codes associated with the device can be employedas well to identify when the device is on or near the surface. Examplesof such non-visual patterns or codes include RFID, short range wirelessradio, capacitive coupling between device and the interactive surface,and an optical connection (e.g., blinking IR LED on bottom of mobiledevice).

According to a second approach, multiple mobile devices can communicateand/or interact with each other via the interactive surface. Initially,each device can establish a connection with the interactive surface.Once connected to the surface, the devices can share data and/orotherwise communicate with one another through the interactive surface.This can be particularly advantageous for sharing information amongdifferent types of mobile devices whether owned by either a single useror multiple users. In some cases, maintaining privacy of someinformation may be a concern for the user. To mitigate this concern, theuser can control which information is openly displayed on theinteractive surface and which information is concealed or hidden from“public” view.

In addition to providing enhanced input/output resources for mobiledevices, the interactive surface can also provide temporary or long-termdata storage for mobile devices, most of which tend to have limitedstorage capacities. Furthermore, the interactive surface allows data orobjects such as files, documents, and photographs to be moved or copiedbetween one or more mobile devices. The interactive surface can alsopermit background monitoring of various tasks or program applicationssuch as email, file transfers, program updates, and the like while theuser is more actively involved in another task or application. Moreover,the interactive surface provides a rich user interface for mobiledevices that are characteristically deficient in display capabilitiesand augments the usability and visibility of such devices.

To the accomplishment of the foregoing and related ends, certainillustrative aspects of the invention are described herein in connectionwith the following description and the annexed drawings. These aspectsare indicative, however, of but a few of the various ways in which theprinciples of the invention may be employed and the subject invention isintended to include all such aspects and their equivalents. Otheradvantages and novel features of the invention may become apparent fromthe following detailed description of the invention when considered inconjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an interactive surface componentinterfacing with one or more mobile devices to facilitate extendinginput/output capabilities for resource deficient mobile devices.

FIG. 2 is a block diagram of an interactive surface componentestablishing a connection with one or more mobile devices to facilitateextending input/output capabilities for resource deficient mobiledevices.

FIG. 3 is a block diagram of exemplary functional interactions betweenmultiple mobile devices, one or more peripheral components, and aninteractive surface component to facilitate transparent user interactionwith the mobile devices.

FIG. 4 is a block diagram of an exemplary interactive surface componentinterfacing with at least one mobile device to facilitate extending theinput/output capabilities of the mobile device.

FIG. 5 is a block diagram of an exemplary interactive surface componentinterfacing with at least one mobile device in a copy-by-occlusionoperation.

FIG. 6 is a block diagram of an exemplary interactive surface componentinterfacing with at least one mobile device in an objectsharing-transfer operation.

FIG. 7 is a block diagram of an exemplary interactive surface componentinterfacing with at least one mobile device whereby users can choose tokeep their data private or hidden from other users interacting with thesurface component.

FIG. 8 is a block diagram of at least two mobile devices otherwiseunable to communicate with each other which are interacting with eachother via an interactive surface component.

FIG. 9 is a block diagram of an exemplary connection scheme in whichmultiple mobile devices can connect to an interactive surface componentto maintain a physical or remote connection with the surface component.

FIG. 10 is a block diagram of an exemplary interactive surface componentinterfacing with at least one group of mobile devices as determined bythe orientation and/or position of such mobile devices.

FIG. 11 is a flow chart illustrating an exemplary methodology thatfacilitates extending input/output capabilities for resource-poor mobiledevices.

FIG. 12 is a flow chart illustrating an exemplary methodology thatfacilitates enhancing user interaction with one or more mobile devicesthrough the use of an interactive surface.

FIG. 13 facilitates communication including data sharing betweenmultiple mobile devices via the interactive surface component.

FIG. 14 illustrates an exemplary environment for implementing variousaspects of the invention.

DETAILED DESCRIPTION

The subject systems and/or methods are now described with reference tothe drawings, wherein like reference numerals are used to refer to likeelements throughout. In the following description, for purposes ofexplanation, numerous specific details are set forth in order to providea thorough understanding of the systems and/or methods. It may beevident, however, that the subject systems and/or methods may bepracticed without these specific details. In other instances, well-knownstructures and devices are shown in block diagram form in order tofacilitate describing them.

As used herein, the terms “component” and “system” are intended to referto a computer-related entity, either hardware, a combination of hardwareand software, software, or software in execution. For example, acomponent may be, but is not limited to being, a process running on aprocessor, a processor, an object, an executable, a thread of execution,a program, and a computer. By way of illustration, both an applicationrunning on a server and the server can be a component. One or morecomponents may reside within a process and/or thread of execution and acomponent may be localized on one computer and/or distributed betweentwo or more computers.

The subject systems and/or methods can incorporate various inferenceschemes and/or techniques in connection with recognizing and identifyingmobile devices and ritualistic or routine interactions there-between.For example, an exemplary interactive surface component can learn toperform particular actions or display certain information when one ormore particular mobile devices connect to the surface component. Inpractice, for instance, imagine that when a smartphone device namedBOBSTAR establishes a connection to the surface, the surface is trainedto automatically show the address book in its display space. Similarly,when PDA-1 and PDA-101 reconnect to the interactive surface component,their state of activity can resume from their previous connection.

As used herein, the term “inference” refers generally to the process ofreasoning about or inferring states of the system, environment, and/oruser from a set of observations as captured via events and/or data.Inference can be employed to identify a specific context or action, orcan generate a probability distribution over states, for example. Theinference can be probabilistic—that is, the computation of a probabilitydistribution over states of interest based on a consideration of dataand events. Inference can also refer to techniques employed forcomposing higher-level events from a set of events and/or data. Suchinference results in the construction of new events or actions from aset of observed events and/or stored event data, whether or not theevents are correlated in close temporal proximity, and whether theevents and data come from one or several event and data sources.

Referring now to FIG. 1, there is a general block diagram 100 of theinteraction between one or more mobile devices 110 and an interactivesurface component 120. The one or more mobile devices 110 can includebut is not limited to a cell phone, smartphone, PDA, tablet PC, laptopcomputer, watch, wearable computing device, and/or memory card. Theinteractive surface component 120 can be oriented horizontally (e.g.,tabletop) or vertically and include a processor (not shown) as well as adisplay screen 130 that provides a rich user interface and asubstantially larger display space than the mobile device 110. Acommunication component 140 creates and/or controls a portal or channelthat allows the interactive surface component 120 to “talk” to themobile device 110 when a connection between them is initiated.

The mobile device 110 often lacks adequate display space or inputcapabilities. However, when the mobile device 110 establishes aconnection (e.g., WiFi, infrared, etc.) with the interactive surfacecomponent 120, the display space of the interactive surface component120 can act as an additional input/output resource for the device 110.In particular, information displayed on the mobile device can bemirrored and enhanced and/or enlarged on the surface component 120. As aresult, a user can more readily and with greater ease enter and/ormanipulate information on his/her mobile device 110 by way of the largerdisplay space and additional input capabilities of the surface component120.

Connection to the surface component 120 can be established in a wirelessmanner such as by contacting the device to at least a portion of thesurface component 120 or by placing the device 110 on the surfacecomponent 120. When the appropriate contact is made, the surfacecomponent 120 can read an identification marker on the device 110 toidentify the type and/or name of the device 110. The identificationmarker can be a barcode, an RFID tag, or some other visual pattern orcode unique to the particular mobile device that can be read by thesurface component 120.

As demonstrated in the later figures, a PDA or tablet PC placed on thesurface 120, for example, can use the surface 120 as a personal portalinto the data displayed on the PDA or tablet PC. This portal provides anadditional parallel plane of interaction for the user. When multipledevices and multiple users are connected to the surface component 120,the position and orientation of each device 110 can be sensed and usedto determine where each different user is located (e.g., sitting) aroundthe surface 120. In practice, for instance, the position and orientationof the device's identification marker can be employed to determine thedevice's overall position and orientation on the surface relative toother devices located thereon.

Referring now to FIG. 2, there is illustrated a block diagram of asystem 200 that facilitates communication between multiple mobiledevices and optionally, one or more input/output components by way of aninteractive surface. The system 200 can include one or more mobiledevices 210 that desire a connection with an interactive surfacecomponent 220. To fully establish communication between the device 210and the surface 220, the interactive surface component 220 can employ aregistration component 230 that identifies the type and/or name of themobile device(s) 210 and calls the appropriate operating protocols foreach device 210. The registration component 230 can also extract theinformation currently displayed on the mobile device 210 and projectsuch information on the interactive surface component 220. In the end,the connection and interaction between the interactive surface component220 and the device 210 are transparent to the user.

When a plurality of mobile devices 210 are connected to the interactivesurface component 220 and interaction between at least two devices 210is desired, an association component 240 can be employed to facilitatethe passage of data from one device to another via the interactivesurface component 220. More specifically, the association component 240can group related devices together or suggest such groupings to theuser(s). The relatedness of devices can be determined in part byexamining their spatial relationships and physical proximity to eachother. For example, devices belonging to the same group are more likelypositioned closer together. It should be appreciated that the at leasttwo devices can be different, alike, or even identical.

The association component 240 can also be employed to couple or link atleast one mobile device 210 with one or more input/output components 250also connected to the interactive surface component 220. Examples ofinput/output components 250 include but are not limited to a keyboard,mouse, stylus, pen, and/or printer. Because most if not all mobiledevices have limited input capabilities, a user may desire to make useof the richer and larger display space of the interactive surfacecomponent to enter data into the device. Therefore, the user may desireto connect a wireless keyboard to the interactive surface component 220.If multiple devices are connected to the interactive surface component220, the keyboard can be readily associated and/or employed with suchdevices without needing to perform complicated pairing procedures eachtime the device or user changes. To mitigate security concerns whenusing such input components, on-screen buttons or controls located onthe interactive surface component 220, and/or on the devices themselves,may be utilized to indicate which device 210 is currently linked to therespective input component 260.

Referring now to FIG. 3, there is a block diagram of an interactivesurface component 300 interfacing with at least a first 310 and a second320 mobile device as well as one or more peripheral input/output devices330. When multiple mobile devices are connected to the interactivesurface component 300, a variety of components can be utilized to assistin the interaction between such mobile devices. The interactive surfacecomponent 300 provides a working medium to bind together multipledevices regardless of their make or model. Virtually any device withsome form of connectivity to the interactive surface component 300 canparticipate and be bound together via a device grouping component 340.The device grouping component can bind multiple heterogeneous devicesusing at least one of the following: a unique visual pattern oridentification marker on the device, physical proximity, alignment ofprimary orientation vectors, and/or explicit controls located on theinteractive surface. For example, the unique visual pattern (e.g.,barcode) can be employed to bind an object (e.g., keyboard, mobiledevice, etc.) seen on the interactive surface to the wirelessinformation being received. Regarding physical proximity, devices placedclose together can be grouped, connected, or otherwise bound in some wayto each other for communication or data sharing purposes.

Next, imagine that a mouse, keyboard, and PDA are all placed on theinteractive surface component 300 and all face the same direction. Basedon the alignment of their primary orientation vectors, these threedevices can be bound together and treated as a group. As a result, theuser can have high-bandwidth text entry on his/her PDA.

When multiple devices are connected to the interactive surfacecomponent, data can be shared or moved between such devices by way of anobject sharing component 350. For example, data such as files,documents, or photographs can be moved or copied to the interactivesurface from the first device 310. Using a navigation component 360 suchas on-screen controls (e.g., touch-sensitive surface, command buttons,etc.), the data can be moved or copied to the second device 320 asdesired by the user. Alternatively, the data can be stored on theinteractive surface component 300 for later retrieval and use.

In general, the navigation component 360 can assist the user in viewingand manipulating the relevant data on the interactive surface component300. That is, the user can navigate through information displayed on theinteractive surface component as if he/she were navigating through thesame information on the mobile device to create a seamless andtransparent experience for the user.

Turning now to the next several figures, exemplary views of theinteractive surface component interfacing with one or more mobiledevices as described hereinabove are illustrated. For instance, FIG. 4depicts the interactive surface component as employed as an additionalinput/output resource for any mobile device that has little or nodisplay space. As shown in this figure, the mobile device 400 is placedon the surface component (in the display space) and after a connectionis established to the surface component, the display of the mobiledevice 400 can be projected onto a portion of the surface display space410 (420) or can consume substantially all of the display space 410(430). When more than one mobile device is connected to the interactivesurface component, the surface display space can be divided such thateach device occupies a particular portion of the display space 410 todisplay their content. The surface display space can mirror the devicedisplay and/or offer enhanced customization and visualization controlsto manipulate data intended for the device. Hence, the surface componentprovides a direct link to the user interface of the device.

In practice, for example, a user could place a cell phone on theinteractive surface component (display space). When a connection isestablished to the surface component, the cell phone could automaticallybe transitioned to speakerphone mode for the duration of the connectionor until the user turns it off or manually switches the mode. Thedisplay space could then show a list of contacts with a picture for eachcontact whereby touching a picture initiates a call or another actionspecific to that person (e.g., opening a shared web browser on theinteractive surface).

As another example, the user could establish a connection betweenhis/her watch and the interactive surface component. When connected, thesurface display space can show various forms to the user to customizethe information which the watch intends to display. Thus, the user canachieve customization of his/her watch using the enhanced controls ofthe interactive surface component rather than dealing with animpoverished display and a paucity of buttons on the watch to achievethe same task.

It should be appreciated that the view of information in the surfacedisplay space can change to present various options for interaction withthe device. Therefore, the information in view in the surface displayspace may not always mirror the mobile device display. When the deviceis removed or picked up from the surface component, the connection canbe severed automatically as a result and the display on the mobiledevice can stay the same or return to a default or standby appearance.In some instances, the user can be asked to verify whether disconnectionis desired before the connection is severed.

Moving on to FIG. 5, a data sharing operation that can be performed by amobile device is demonstrated. In particular, a user places his/herdevice 500 on the surface display space 410 of an interactive surfacecomponent and establishes communication therewith. In the surfacedisplay space 410, there is a plurality of data objects: A, B, and C,for example. These objects may have been put there by another mobiledevice (not shown) or by the device 500 at an earlier time. Suppose thatthe user now desires to add object A to his/her device 500. To do so,the user can put or drag the device 500 over the desired object(s). Copyby occlusion is performed when object A is occluded by the device 500.An on-screen command/button can be invoked as well. Placing the device500 over the desired object (with or without invoking the command)essentially moves or copies the object(s) to the device 500. This can beaccomplished in part by the use of invisible or visible tags on theobjects. For example, each object can include a tag or marker which mustbe “scanned” or otherwise “hit” by the device in order for the object tobe copied or moved to the device. When dealing with a group of objects,each object need not be occluded separately. Rather, the group can bemoved or copied as a whole when occluded or when the group's tag ormarker is somehow hit by the device 500.

Objects can be moved or copied to the device 500 from “the interactivesurface” 410 or moved or copied from the device 500 to the surface 410via the device's clipboard 602. In FIG. 6, a mobile device's clipboardor data store 602 can be displayed in the surface display space 410,next to the respective device 500. By displaying the device's clipboard602 in this manner, a user or other users viewing the surface displayspace 410 can drag, move, or copy objects in and out of the clipboard602. To further facilitate the movement of objects into and out of theclipboard 602, in/out targets on the clipboard 602 can be employed toexplicitly indicate the desired placement of the object.

FIG. 7 builds upon the clipboard aspect of FIG. 6. More specifically,users can keep the contents of their clipboard or data store private,hidden, and/or even separate from other users' data stores that are alsointeracting with the surface display space in some manner. As shown inthe figure, shared objects 700 can be maintained in the surface displayspace whereas personal objects are maintained on the appropriate mobiledevices—via their private stores 710. This mitigates confusion whencollaborating or sharing information among multiple users and/ormultiple devices.

The diagram 800 in FIG. 8 illustrates that multiple devices, asillustrated by devices 802 and 804, for example, which are otherwiseunable to communicate (e.g., share data) with each other but can nowcommunicate and interact with each other via the interactive surfacecomponent. For communication between devices to be successful, bothdevices are connected to the interactive surface component. Theconnection can be wireless and in some cases, may even be remote. Thatis, one or more of the devices are not physically present but rathervirtually present on the interactive surface. Multiple devices canconnect to the interactive surface in a number of ways. One such way isthrough the use of explicit surface controls as demonstrated in FIG. 9,infra.

FIG. 9 demonstrates how multiple devices (e.g., a tablet PC 902 andsmartphone 904 are depicted but could also be only tablet PCs, acombination of any other devices (at least two), etc.) can becomeconnected using the interactive surface component as a medium to enablesuch devices to learn or determine the proximity of other devices andtheir spatial relationship to those devices. First at 910, a firstdevice is placed on the interactive surface (display space 410). Theinteractive surface shows a “cord” coming out of the device toillustrate the potential for connection. The “cord” can be graphicallyrendered on the interactive surface when the presence of the firstdevice is detected. At this point, no other device including the firstdevice is connected yet as indicated by the “white” connection target906.

At 920, a second device (W) 904 is put down on the interactive surface(display space 410). A connection between the two devices (902 and 904)can be hypothesized by showing both cords connected to the connectiontarget 906 (shaded to indicate a connection). The user can touch thetarget area 906 or some other symbol, icon, or command to connect thedevices, or touch and hold it (e.g., target area 906, symbol, icon,command button, etc.) to bring up other options.

At 930, both the first and second devices (902 and 904) are connected toeach other. If one of the devices is removed from the interactivesurface (display space 410), the connection can be automatically broken.Alternatively, the connection can persist when the second device hasbeen physically removed, as is illustrated at 930. This can allow theuser to pick up his/her device and use it as a private display, or evento wander about at some distance from the interactive surface whilestaying connected to his/her colleagues positioned at the interactivesurface. When the second device has been moved some distance away, suchas down the hall or to another room, a virtual indication 908 of theconnection to the second device W 904 can be shown on the surface, andthe second device can communicate with the surface component and/or anyother connected devices remotely. The surface can display an “antennatower” 912 to provide feedback of the ongoing connection, and aconnection icon can give local users the option to close the connectionto the second device W.

While the second device W 904 is away or interacting remotely, the localscreen on W 904 can show similar feedback indicating the connection tothe surface (display space 410) (and the first device) as well as theoption to close that connection. In addition, the local screen on thedevice can be manipulated to reflect or clarify any activity occurringon the interactive surface. For instance, points of connection can bedisplayed on the mobile device to indicate that they are available tothe device. A matching connection on the surface (display space 410)could tie directly into that point graphically (on the same side of thedisplay, corner, etc.).

As information is being communicated between devices via the surface,such as between the first and second devices, one or more graphics orgraphical illustrations can be employed to visually indicate the statusof activity between at least two devices. For example, the connectionbetween any two devices can glow or pulse to depict the flow of dataand/or the current state of the connection. This can assist the user(s)in determining the quantity or type of information that has been sharedor passed, time remaining to complete the transfer, and/or time requiredto complete the transfer. Graphics displayed on the interactive surfacecan also be utilized to terminate, reassign, or add connections betweendevices. As a result, the user can interact with the relevantconnection(s) directly rather than through the various devices.

Turning now to FIG. 10, the diagram 1000 illustrates that one or moreobjects (shared objects 1002) can be moved or copied to a group ofdevices (J's devices 1004) in a single transaction instead of moving orcopying each object to each device within the group via separatetransactions. Devices can be grouped according to their relativeposition and orientation on the interactive surface (display space 410),and such devices can be bound to each other. For example, those devicespositioned more closely together and/or oriented in the most similarmanner can be implicitly or explicitly grouped together by theinteractive surface. For example, three groups are illustrated. K'sdevices, which are positioned closely together and oriented in a similarmanner, are shown in group 1006, P's devices, which are positionedclosely together and oriented in a similar manner, are shown in group1008, and as mentioned previously, J's devices are shown in group 1004.When one device is removed from the group and disconnected from thesurface, the surface can remember that such device belonged to aparticular group. Thus, when that device is reconnected to the surfaceand at least one other device from the group is still connected, theinteractive surface can automatically associate the reconnected deviceto the group (and/or any other devices within the group).

When multiple devices are present on the interactive surface butcommunication is only desired with one particular device, the spatialrelationship between such devices can assist in identifying the desireddevice. In practice, for example, imagine that JOHN wants to sharepictures with JANE. Their devices as well as 2 others belonging to otherusers are connected to the interactive surface. To facilitate theexchange of the pictures between only JOHN's and JANE's devices, theycan point their devices to face each other. Alternatively, they canorient them in a similar manner and within close proximity of eachother. Other variations are possible as well to indicate thatcommunication between only the designated devices is desired. Commandsor additional controls associated with the interactive surface may alsobe employed.

The interactive surface also remembers which devices have previouslybeen placed upon it, and what tasks or files were used among thedevices. If the same set of devices is placed on the interactive surfaceagain, the surface may offer to restore the old state of the surface.This allows several colleagues to temporarily stop working on a project,and then return to it later just by placing their same devices on thetable. A single user could similarly restore the workspace for a projectjust by placing a tagged paper folder or electronic device usedspecifically for that project on the table. For example, placing an MP3player on the table would bring up the user's music files and web siteshe/she had last used for sharing and purchasing music.

Devices on the interactive surface and connected thereto (e.g., thedevices of groups 1004, 1006, or 1008) can also operate as storagecontainers for data and proxies similar to network folders or shares.For instance, a PDA can be a proxy that points to a URL for a networkshare. In this type of arrangement of devices on the interactivesurface, the interactive surface can facilitate visualization of awireless network.

The interactive surface can also serve as an extended workspace area fora user who desires additional screen real estate. For example, the usercan connect his tablet PC to the interactive surface and create aworkspace arranged in the surface display space around the tablet PC.The user can use various hand gestures or other navigation controls tobring new documents onto the tablet screen (or remove them from it).This alleviates screen real estate restrictions on the tablet whenworking with lots of documents. The interactive surface provides a largeperipheral view of backburner tasks and programs like email that theuser can monitor even when not directly using them.

Mobile devices such as a tablet PC, for instance, can also be placed ina slate mode on the interactive surface to provide an additionalhorizontal layer. The user can dynamically transition to a verticalsurface by flipping up the screen to be perpendicular to the surface.This can be sensed by the surface which can give a separate view that isprivate. Alternatively or in addition, the surface might switch from anoverhead plan view on the tablet PC to a 3D immersive view. To mitigateundesired occlusion of any portion of the display space, the tablet PCcan be placed near the edges of the display space.

Various methodologies will now be described via a series of acts. It isto be understood and appreciated that the subject system and/ormethodology is not limited by the order of acts, as some acts may, inaccordance with the subject application, occur in different ordersand/or concurrently with other acts from that shown and describedherein. For example, those skilled in the art will understand andappreciate that a methodology could alternatively be represented as aseries of interrelated states or events, such as in a state diagram.Moreover, not all illustrated acts may be required to implement amethodology in accordance with the subject application.

Referring now to FIG. 11, there is a flow diagram of an exemplary method1100 that facilitates extending input/output capabilities forresource-poor mobile devices. The method 1100 involves establishing awireless connection between at least one mobile device and aninteractive surface component at 1110. The mobile device can be any oneof a cell phone, smartphone, PDA, tablet PC, laptop computer, watch,wearable computing device, memory stick, and/or memory card. Theinteractive surface component can be oriented in a horizontal mannersimilar to a table or in a vertical manner.

At 1120, the at least one mobile device can be identified by way of itsunique visual pattern (e.g., barcode, RFID tag, etc.). Once identifiedby name and/or device type (make and/or model), the interactive surfacecomponent can determine the position and orientation of the mobiledevice as it rests on its surface and can register the device's userinterface and any content appearing thereon at 1130. At 1140, thecontents of the mobile device display can be visualized in the displayspace of the interactive surface component to facilitate transparentuser interaction with the interactive surface. In other words, themobile device display can be mirrored on the larger and richer userinterface of the interactive surface component. As a result, the usercan interact with his/her mobile device as usual with the assistance ofthe expanded input/output capabilities of the interactive surface.

Referring now to FIG. 12, there is illustrated a flow diagram of anexemplary method 1200 that facilitates enhancing user interaction withone or more mobile devices through the use of an interactive surface.The method 1200 involves detecting the presence of at least one mobiledevice within some distance of the interactive surface component at1210. For example, detection can occur when the device is placed on theinteractive surface or when the device is within a physical proximity ofthe interactive surface. Once detected, the visual identification codeor pattern on the device can be read at 1220. By doing so, theinteractive surface can determine the most appropriate protocols to useto communicate and/or interact with the mobile device.

At 1230, the interactive surface can project one or more suitabledisplays specific to the mobile device onto the display space of theinteractive surface. For instance, the user or the device itself mayhave set a default screen to view when initially connected to theinteractive surface. At 1240, the interactive surface can provideenhanced customization and/or visualization controls particularly formobile devices with little or no display capabilities or inadequateinput or navigation controls or for users who desire additionalworkspace. At 1250, the device can be optionally removed from thesurface to terminate or end communication and interaction with theinteractive surface. Termination can be automatic or can beaffirmatively verified by the user before the connection is ended. Whentransferring or sharing data between the interactive surface and thedevice, the connection can remain open for the duration of the dataexchange and end as soon as the exchange is completed.

In some cases, the device can maintain a virtual presence on theinteractive surface and communicate with the interactive surface from aremote location. This can provide users with some flexibility wheninteracting with the interactive surface so that they do not always needto disconnect and then re-connect if they need to step away from theinteractive surface with their device for a brief time. The remoteconnection can be terminated from either the interactive surface or fromthe remote device.

Turning now to FIG. 13, there is illustrated a flow diagram of anexemplary method 1300 that facilitates communication including datasharing between multiple mobile devices via the interactive surfacecomponent. The method 1300 involves connecting a first device to theinteractive surface at 1310 and determining the position and/ororientation of the first device at 1320. At 1330, at least a seconddevice can be connected to the interactive surface. The position and/ororientation of the second device and any other subsequently added devicecan be determined with respect to the previously added devices at 1340to obtain an overall representation of the spatial relationships betweenthe connected devices.

At 1350, it can be optionally determined whether some devices can orshould be grouped according to their respective spatial arrangements onthe interactive surface. For instance, one user may have two mobiledevices and a wireless keyboard connected to the interactive surface.The interactive surface can suggest a grouping of these devices tofacilitate data sharing and input capabilities. Alternatively, variousteams of users involved on the same project may be interacting with theinteractive surface. Thus, each team of users can have their devicesgrouped together to make data sharing more efficient. At 1360,communication and/or interaction between the multiple devices can beperformed. Examples of such types of interaction include moving andcopying data between devices or groups of devices, viewing data from thedevices on the interactive surface, sharing input devices such as awireless keyboard between different users and for different deviceswithout requiring complex pairing procedures, etc.

In order to provide additional context for various aspects of thesubject application, FIG. 14 and the following discussion are intendedto provide a brief, general description of a suitable operatingenvironment 1410 in which various aspects of the subject application maybe implemented. While the system(s) and/or method(s) is described in thegeneral context of computer-executable instructions, such as programmodules, executed by one or more computers or other devices, thoseskilled in the art will recognize that the invention can also beimplemented in combination with other program modules and/or as acombination of hardware and software.

Generally, however, program modules include routines, programs, objects,components, data structures, etc. that perform particular tasks orimplement particular data types. The operating environment 1410 is onlyone example of a suitable operating environment and is not intended tosuggest any limitation as to the scope of use or functionality of thesystem and/or method. Other well known computer systems, environments,and/or configurations that may be suitable for use with the systemand/or method include but are not limited to, personal computers,hand-held or laptop devices, multiprocessor systems,microprocessor-based systems, programmable consumer electronics, networkPCs, minicomputers, mainframe computers, distributed computingenvironments that include the above systems or devices, and the like.

With reference to FIG. 14, an exemplary environment 1410 forimplementing various aspects of the system and/or method includes acomputer 1412. The computer 1412 includes a processing unit 1414, asystem memory 1416, and a system bus 1418. The system bus 1418 couplessystem components including, but not limited to, the system memory 1416to the processing unit 1414. The processing unit 1414 can be any ofvarious available processors. Dual microprocessors and othermultiprocessor architectures also can be employed as the processing unit1414.

The system bus 1418 can be any of several types of bus structure(s)including the memory bus or memory controller, a peripheral bus orexternal bus, and/or a local bus using any variety of available busarchitectures including, but not limited to, 11-bit bus, IndustrialStandard Architecture (ISA), Micro-Channel Architecture (MCA), ExtendedISA (EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB),Peripheral Component Interconnect (PCI), Universal Serial Bus (USB),Advanced Graphics Port (AGP), Personal Computer Memory CardInternational Association bus (PCMCIA), and Small Computer SystemsInterface (SCSI).

The system memory 1416 includes volatile memory 1420 and nonvolatilememory 1422. The basic input/output system (BIOS), containing the basicroutines to transfer information between elements within the computer1412, such as during start-up, is stored in nonvolatile memory 1422. Byway of illustration, and not limitation, nonvolatile memory 1422 caninclude read only memory (ROM), programmable ROM (PROM), electricallyprogrammable ROM (EPROM), electrically erasable ROM (EEPROM), or flashmemory. Volatile memory 1420 includes random access memory (RAM), whichacts as external cache memory. By way of illustration and notlimitation, RAM is available in many forms such as synchronous RAM(SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rateSDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), anddirect Rambus RAM (DRRAM).

Computer 1412 also includes removable/nonremovable, volatile/nonvolatilecomputer storage media. FIG. 14 illustrates, for example a disk storage1424. Disk storage 1424 includes, but is not limited to, devices like amagnetic disk drive, floppy disk drive, tape drive, Jaz drive, Zipdrive, LS-100 drive, flash memory card, or memory stick. In addition,disk storage 1424 can include storage media separately or in combinationwith other storage media including, but not limited to, an optical diskdrive such as a compact disk ROM device (CD-ROM), CD recordable drive(CD-R Drive), CD rewritable drive (CD-RW Drive) or a digital versatiledisk ROM drive (DVD-ROM). To facilitate connection of the disk storagedevices 1424 to the system bus 1418, a removable or non-removableinterface is typically used such as interface 1426.

It is to be appreciated that FIG. 14 describes software that acts as anintermediary between users and the basic computer resources described insuitable operating environment 1410. Such software includes an operatingsystem 1428. Operating system 1428, which can be stored on disk storage1424, acts to control and allocate resources of the computer system1412. System applications 1430 take advantage of the management ofresources by operating system 1428 through program modules 1432 andprogram data 1434 stored either in system memory 1416 or on disk storage1424. It is to be appreciated that the subject system and/or method canbe implemented with various operating systems or combinations ofoperating systems.

A user enters commands or information into the computer 1412 throughinput device(s) 1436. Input devices 1436 include, but are not limitedto, a pointing device such as a mouse, trackball, stylus, touch pad,keyboard, microphone, joystick, game pad, satellite dish, scanner, TVtuner card, digital camera, digital video camera, web camera, and thelike. These and other input devices connect to the processing unit 1414through the system bus 1418 via interface port(s) 1438. Interfaceport(s) 1438 include, for example, a serial port, a parallel port, agame port, and a universal serial bus (USB). Output device(s) 1440 usesome of the same type of ports as input device(s) 1436. Thus, forexample, a USB port may be used to provide input to computer 1412 and tooutput information from computer 1412 to an output device 1440. Outputadapter 1442 is provided to illustrate that there are some outputdevices 1440 like monitors, speakers, and printers among other outputdevices 1440 that require special adapters. The output adapters 1442include, by way of illustration and not limitation, video and soundcards that provide a means of connection between the output device 1440and the system bus 1418. It should be noted that other devices and/orsystems of devices provide both input and output capabilities such asremote computer(s) 1444.

Computer 1412 can operate in a networked environment using logicalconnections to one or more remote computers, such as remote computer(s)1444. The remote computer(s) 1444 can be a personal computer, a server,a router, a network PC, a workstation, a microprocessor based appliance,a peer device or other common network node and the like, and typicallyincludes many or all of the elements described relative to computer1412. For purposes of brevity, only a memory storage device 1446 isillustrated with remote computer(s) 1444. Remote computer(s) 1444 islogically connected to computer 1412 through a network interface 1448and then physically connected via communication connection 1450. Networkinterface 1448 encompasses communication networks such as local-areanetworks (LAN) and wide-area networks (WAN). LAN technologies includeFiber Distributed Data Interface (FDDI), Copper Distributed DataInterface (CDDI), Ethernet/IEEE 1102.3, Token Ring/IEEE 1102.5 and thelike. WAN technologies include, but are not limited to, point-to-pointlinks, circuit switching networks like Integrated Services DigitalNetworks (ISDN) and variations thereon, packet switching networks, andDigital Subscriber Lines (DSL).

Communication connection(s) 1450 refers to the hardware/softwareemployed to connect the network interface 1448 to the bus 1418. Whilecommunication connection 1450 is shown for illustrative clarity insidecomputer 1412, it can also be external to computer 1412. Thehardware/software necessary for connection to the network interface 1448includes, for exemplary purposes only, internal and externaltechnologies such as, modems including regular telephone grade modems,cable modems and DSL modems, ISDN adapters, and Ethernet cards.

What has been described above includes examples of the subject systemand/or method. It is, of course, not possible to describe everyconceivable combination of components or methodologies for purposes ofdescribing the subject system and/or method, but one of ordinary skillin the art may recognize that many further combinations and permutationsof the subject system and/or method are possible. Accordingly, thesubject system and/or method are intended to embrace all suchalterations, modifications, and variations that fall within the spiritand scope of the appended claims. Furthermore, to the extent that theterm “includes” is used in either the detailed description or theclaims, such term is intended to be inclusive in a manner similar to theterm “comprising” as “comprising” is interpreted when employed as atransitional word in a claim.

The invention claimed is:
 1. A method comprising: identifying a mobiledevice that is placed on an interactive surface component, the mobiledevice including a display displaying content and the interactivesurface component comprising an interactive surface component displaylarger than a display of the mobile device; establishing a connectionbetween the mobile device and the interactive surface component;displaying, on the interactive surface component display, the contentdisplayed on the mobile device; receiving an input via input controls onthe interactive surface component; based at least in part on the input,manipulating the display on the interactive surface component displayand causing the mobile device to manipulate the content on the mobiledevice; and creating a communication channel between the mobile deviceand a second mobile device via the interactive surface component basedat least in part on the identifying the mobile device that is placed onthe interactive surface component and determining that the second mobiledevice is connected to the interactive surface component.
 2. A method asrecited in claim 1, further comprising calling a protocol correspondingto the mobile device.
 3. A method as recited in claim 1, furthercomprising facilitating data sharing between the mobile device connectedto the interactive surface component as a first identified mobile deviceand a second identified mobile device.
 4. A method as recited in claim3, wherein the first identified mobile device and the second identifiedmobile device are heterogeneous mobile devices.
 5. A method as recitedin claim 1, wherein the identifying the mobile device comprisesdetermining that the mobile device is placed on the display of theinteractive surface component, the method further comprising:determining that the second mobile device is placed on the display ofthe interactive surface component; and based at least in part on thedetermining that the mobile device is placed on the display of theinteractive surface component and the determining that the second mobiledevice is placed on the display of the interactive surface component,creating the communication channel between the mobile device and secondmobile device.
 6. A method as recited in claim 1, further comprisingdetermining a spatial relationship between the mobile device and thesecond mobile device connected to the interactive surface component. 7.A method as recited in claim 6, further comprising, at least in partbased on the spatial relationship, grouping the mobile device and thesecond mobile device into a communication group.
 8. A method as recitedin claim 1, further comprising copying data displayed on the interactivesurface component display to the mobile device responsive, at least inpart, to the mobile device being placed over the data.
 9. A method asrecited in claim 1, further comprising causing the interactive surfacecomponent to display a display workspace for the mobile device that isplaced on the interactive surface component display when the mobiledevice contacts the interactive surface component, the display workspacefacilitating background monitoring via the display on the interactivesurface component.
 10. A method as recited in claim 1, furthercomprising sharing an input device between the mobile device as a firstmobile device and the second mobile device without requiring a pairingprocedure.
 11. A system comprising: an interactive surface componentcomprising an interactive surface component display; one or moreprocessors; and memory storing one or more computer-executableinstructions, the instructions being executed by the one or moreprocessors to perform operations comprising: identifying a mobile devicethat is placed on the interactive surface component; establishing acommunication channel between the mobile device and the interactivesurface component to control interaction between the mobile device andthe interactive surface component; displaying content from the mobiledevice on the interactive surface component display; receiving an inputvia input controls on the interactive surface component; manipulatingthe content displayed on the interactive surface component display basedat least in part on the input; causing the mobile device to manipulatethe content on the mobile device based at least in part on the input;and creating a communication channel between the mobile device and asecond mobile device via the interactive surface component in responseto identifying that the mobile device is placed on the interactivesurface component and determining that the second mobile device isconnected to the interactive surface component.
 12. A system as recitedin claim 11, the instructions being executed by the one or moreprocessors to perform further operations comprising: identifying thesecond mobile device; and providing information to at least one ofmobile device or the second mobile device to facilitate data sharingamong the mobile devices connected to the interactive surface componentas a first mobile device and the second mobile device.
 13. A system asrecited in claim 12, wherein the first mobile device and the secondmobile device are heterogeneous mobile devices.
 14. A system as recitedin claim 12, the instructions being executed by the one or moreprocessors to perform further operations comprising providinginformation to at least one of the first mobile or the second mobiledevice to connect via at least one communication channel when the firstmobile device and the second mobile device are placed on the interactivesurface component.
 15. A system as recited in claim 11, wherein aninterface for connecting the mobile device to the interactive surfacecomponent includes a display component.
 16. A computer storage memory ormedia device having computer-executable instructions recorded thereon,the computer-executable instructions upon execution by a processor,programming an interactive surface component to perform operationscomprising: establishing a connection between a mobile device and theinteractive surface component, the interactive surface componentincluding a display space that is larger than a display of the mobiledevice; displaying data from the mobile device as a display on thedisplay space of the interactive surface component; configuring inputcontrols on the interactive surface component to receive inputs tomanipulate the display; receiving, via the input controls, an input tomanipulate the display; manipulating the display based at least in parton the input; causing the mobile device to manipulate the data on themobile device based at least in part on the input; and creating acommunication channel between the mobile device and a second mobiledevice via the interactive surface component in response to identifyingthat the mobile device is placed on the interactive surface componentand determining that the second mobile device is connected to theinteractive surface component.
 17. A computer storage memory or mediadevice as recited in claim 16, the operations further comprising callinga protocol corresponding to the mobile device.
 18. A computer storagememory or media device as recited in claim 16, the operations furthercomprising facilitating data sharing among the mobile devices connectedto the interactive surface component as a first identified mobile deviceand the second identified mobile device.
 19. A computer storage memoryor media device as recited in claim 18, wherein the first identifiedmobile device and the second identified mobile device are heterogeneousmobile devices.
 20. A computer storage memory or media device as recitedin claim 16, the operations further comprising receiving an indicationthat the mobile device is disposed on the display space, wherein theestablishing the connection is performed at least in part in response toreceiving the indication and the connection is a wireless connection.